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Laser Rangefinder (Time-of-Flight, Electromagnetic) 

► It achieves significant improvements over the ultrasonic range 
sensor owing to the use of laser light instead of sound. 

► It is often referred to as optical radar or lidar (Light 
Detection And Ranging). 

► These devices produce a range estimate based on the time 
needed for the light to reach the target and return. 

► A mechanical mechanism with a mirror sweeps the light beam 
to cover the required scene in a plane (2D) or even in three 
dimensions, using a rotating, nodding mirror (3D). 
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Laser Rangefinder (cont.) 
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Laser Rangefinder (cont.) 



► This type of sensor consists of: 

► A transmitter which illuminates a target with a collimated beam 
(e.g., laser). 

► A receiver capable of detecting the component of light which is 
essentially coaxial with the transmitted beam. 

► Receiver detects the time needed for round-trip. 
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Laser Rangefinder (cont.) 




Operating Principles: 

► Use a pulsed laser and then measure the elapsed time directly, 
just as in the ultrasonic, (capable of resolving picoseconds) 
(today the standard). 

►• Measure the beat frequency between a frequency-modulated 
continuous wave (FMCW) and its received reflection. 

► Measure the phase shift of the reflected light. 
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Phase-Shift Measurement 
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Transmitter 



Phase 
Measurement 
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Transmitted Beam 
Reflected Beam 
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Where: 

c: is the speed of light; Hhe modulating frequency; D'the distance covered by the 
emitted light is. 

■ for f = 5 MHz (as in the A.T&T. sensor), X - GO meters 
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Phase-Shift Measurement (cont.) 

► Near-infrared light from a light-emitting diode [LED] or laser. 

► Diffuse reflection will occur for surfaces having a roughness 
greater than the wavelength of the incident light. 

► The wavelength of the infrared light emitted is 824 nm and so 
most surfaces, with the exception of only highly polished 
reflecting objects, will be diffuse reflectors. 

► The sensor transmits 100% amplitude modulated light at a 
known frequency and measures the phase shift between the 
transmitted and reflected signals. 
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Phase-Shift Measurement (cont.) 



■ Distance D. between the beam splitter and the target 

■ where 

■ G: phase difference between transmitted and reflected beam 

■ Theoretically ambiguous range estimates 

- since for example if A. — 60 meters, a target at a range of 5 meters — target at 35 




Transmitted Beam 
Reflected Beam 
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Phase-Shift Measurement (cont.) 



1 Uncertainty of the range (phase/time estimate) is inversely proportional t 
the square of fhe received signal amplitude. 

- Hence dark, distant objects will not produce such good range estimated as 
closer brighter objects ... 
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Phase-Shift Measurement (cont.) 



■ Typical range image of a 2D laser range sensor with a rotating mirror. The length of 
the lines through the measurement points indicate the uncertainties. 
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The SICK LMS 200 Laser Scanner 



■ Angular resolution 0.25 deg 

■ Depth resolution ranges between 1 and 1 5 mm and the typical accuracy 
is 35 mm, over a range from 5 cm up to 20 m or more (up to 80 m), 
depending on the reflectivity of the object being ranged. 

■ This device performs seventy five 180-degrees scans per second 
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Note 1 



Laser rangefinders cannot detect the presence of optically 
transparent materials such as glass, and this can be a significant 
obstacle in environments. 



trasonic ranging sensors, an important error mi 

'. Vith light, thi 
i striking 



ily 



,3, gz ; 



CIS Faculty 
ient of Information Systems 



Mansoura University 




Note 1 



Laser rangefinders cannot detect the presence of optically 
transparent materials such as glass, and this can be a significant 
obstacle in environments. 



Note 2 



As with ultrasonic ranging sensors, an important error mode 
involves coherent reflection of the energy. With light, this will only 
occur when striking a highly polished surface. 
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3D Laser Range Finder 



A 3D laser range finder is a laser scanner thai acquires scan data in more than a single 
plane. 

Custom-made 3D scanners are typically built by nodding or rotating a 2D scanner in a 

stepwise or continuous manner around an axis parallel to the scanning plane. 

By lowering the rotational speed ot the turn-table ; the angular resolution in the horizontal 

direction can be made as small as desired. 

A full spherical field of view can be covered (360 ° in azimuth and +/-90 ° in elevation). 

However, acquisition takes up to some seconds! 

For instance, if our laser takes 75 plane-scans/sec and we need an azimuthal angular 

resolution of 0.25 degrees, the period For a half rotation of the turn-table necessary to capture 

a spherical 3D scan with two Sicks is then 360 I 0.25 / 75 / 2 = 9.6 seconds. If one is satisfied 

with an azimuthal angular resolution of 1 degree, then the acquisition time drops down to 2.4 

seconds, which is still too high for 3D mapping during motion! 
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3D Laser Range Finder (cont.) 



1 The Alasca XT laser scanner splits the laser beam into four vertical 

layers with an aperture angle of 3.2°. 
1 This sensor is typically used for obstacle and pedestrian detection on cars. 
Because of its multi-layer scanning principle, it allows us any pitching of 
the vehicle 
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3D Range Sensor: Time Of Flight (TOF) camera 



A Time-of-Flight camera (TOF camera, figure } works similarly to a lidar with the advantage that the whole 
3D scene is captured al the same time and that there are no moving parts. This device uses a 

modulated infrared lighting source to determine the distance for each pixel of a Photonic Mixer Device 
(PMD) sensor. 




Swiss Ranker 30Q0 
(produced by MESA] 
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Incremental Object Part Detection 
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Range Camera 

- 3D information with high data rate (100 Hz) 

- Compact and easy to manage 

- High, non-uniform measurement noise 

■ High outlier rate at jump edges 

■ However very low resolution (174x144 pixels) 






3D Point CLoud 



Range Camera SR- 3-000 
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Triangulation Ranging 

• Use of geometrical properties of the image to establish a distance measurement 

■ If a well defined light pattern (e.g. point, line) is projected onto the environment. 

■ reflected light is then captured by a photo-sensitive line or matrix (camera) sensor device 

■ simple triangulation allows to establish a distance, 

■ If size of a captured object is precisely known 

■ triangulation without light projecting 
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Laser Triangulation (ID) 
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Principle of 1 D laser triangulation : /> = f — 
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e Ranging Sensors (P2) 
Vision 

Overview 
Pinhole Model 
Stereo Vision 
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One picture, a thousand words 



■ Of all our senses, vision is the most powerful in aiding our perception of 
the 3D world around us. 

■ Retina is ~1000m 2 _ Contains millions of photoreceptors 

(120 mil. rods and 7 mil. Cones for colour sampling) 

■ Provides enormous amount of information: data-rate of ~3GBytes/s 
■=> a large proportion of our brain power is dedicated to processing the 

signals from our eyes 



From Front of the Eye in Back of the Brain 
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Human Visual Capabilities 



Our visual system is very sophisticated 

Humans can interpret images successtully under a wide range ol 

conditions — even in the presence of very limited cues 
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Vision for Robotics 




Enormous descriptability of images 

■=> a lot of data to process (human vision involves 60 billion neurons!) 



Not sensible to copy the biology, but learn from it 



Capture light O Convert to digital image 
■* Process to get 'salient' information 

Vision is increasingly popular as a sensing 
modalily: 
- compactness, 

■ compatibility, 

■ low cost, ... 

■ HW advances necessary to support the 
processing of images < 
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Computer Vision 
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Connection to other disciplines 
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Applications of Computer Vision 




Factory inspection 



Reading license plates, 
checks, ZIP codes 




Surveillance 



Autonomous driving, 
robot navigation 






Monitoring for safety 
(Poseidon) 




Driver assistance 

(collision warning, lane departure 

warning, rear abject detection) 
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Applications of Computer Vision 



Assistive technologies 




Entertainment 
(Sony EyeToy) 



Movie special effects 




Digital cameras (face detection for setting focus. Visual search 

exposure) http://www.kooaba.com/ 
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The camera 
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How do we see the world? 




object 




film 



■ Place a piece of film in front of an object 
■* Do we get a reasonable image? 
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Pinhole camera 




film 



Add a barrier to block off most of the rays 
■ This reduces blurring 
- The opening known as the aperture 
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Pinhole camera model 




Pinhole model: 

■ Captures pencil of rays — all rays through a single point 

■ The point is called Center of Projection or Optical Center 

■ The image is formed on the Image Plane 



ijtjb 



CIS Faculty 
Department of Information Systems 



Mansoura University 




Shrinking the aperture 




Why not make the aperture as small as possible? 

■ Less light gets through (must increase the exposure) 

■ Diffraction effects... 
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Solution: adding a lens 



object 




A lens focuses light onto the film 

« Rays passing through the center are not deviated 
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Solution: adding a lens (cont.) 
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object 




Optical Axis 




jrcle of confusion 

or 

Blur Circle 



1 A lens focuses light onto the film 

■ Rays passing through the center are not deviated 

■ All rays parallel to the Optical Axis converge at the Focal Point 
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Thin lens equation 
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Lens 
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Similar Triangles: 



5 _ e 

A ~ z 
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Thin lens equation (cont.) 




- Similar Triangles 
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Thin lenses 
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1 Thin lens equation: 



1 

7 



■ Any object point satisfying this equation is in focus 

• This formula can also be used to estimate roughly the distance to the object 
("Depth from Focus") 
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In Focus 
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object 




mm 



There is a specific distance at which objects are 'in focus" 
other points project to a "blur circle" in the image 
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Perspective 



► The dependence of the apparent size of objects on their 
distance from the observer is known as perspective. 

► Perspective gives us very strong depth cues ==> hence we can 
perceive a 3D scene by viewing its 2D representation (i.e. 
image). 



► When viewing 3D scenes, it is possible to be mislead by 
perception. 
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Perspective Camera 
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O = principal point 



~£ c = optical axis 
2, 




Image plane (CGD) 



C = optical center = center of the lens 



For convenience, the image plane is usually represented in front so that the 
image preserves the same orientation (i.e. not flipped) 

Notice: a camera does not measure distances but angles! Therefore it is 
a "bearing sensor" 






CIS Faculty 
Department of Information Systems 



Mansoura University 




Perspective Projection 



From the Camera frame to the local image plane 

■ The 3D scene point P=(X C Y c Z C J 

projects to point P = \x y) onto the (local) image plane 



P=(X c ,0, Z C ) T 



• Analysing similar triangles: 
x X c fX c 

f ~ z . X ~ z* 
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■ So. for the 3D case, we can also obtain: 

/ 2. z e 
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Perspective Projection (General) 



Image plane 
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From the Camera frame to pixel coordinates (0,0) 

- To convert the coordinates at p. from the local image plane 
coordinates (x r y) to the pixel coordinates (it.v) we have to take 
account for: 

- Pixel coordinates of the camera optical centerO = («,>, v a ) 

■ Scale factors for the pixel-size in both dimensions^ , k v 

KM 

II = If., + K X => It = M n 4- 

So: Z v 

v — v., -+■ k y => v — v n -n — '■ — 

o Z e 

- Use Homogeneous Coordinates for linear mapping from 3D to 2D, by introducing an extra 
element: 



•■(: 



" An 
Av 
A 



and similarly for the world coordinates. Note, usually A = 1 
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Perspective Projection 




H — H c , + 



V — V, b + 



KfK 



Expressed in matrix form & Homogenerous coords 
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Or alternatively 
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Image plane (COD) 



Focal length in u-direction 

-> Focal length in v-direction 
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Projection onto the image plane 



CIS Faculty 
Department of Information Systems 




From the Camera frame to the World frame 
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[ 
Extrinsic Parameters 



Projection Matrix 



1 — I 



ijLjb 



CIS Faculty flTJ 
Department of Information Systen 



Mansoura University 



Radial distortion 




Straight lines in the world ■=> Straight lines in the image? 



From ideal fn,f) to distorted 
pixel coordinates (tt rf , i\.,l 

Simple distortion model: 



= <i+i/c,y-> 



IS — U. 
v — v r 



Radial Distortion 
parameter 




Barrel distortion Pincushion distortion 



r = iu-it Q ) +- (v — v„ ) 
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Camera Calibration 



Measure all unknown parameters to form the camera model {intrinsic & extrinsic) 
■=> use model to 'understand' the projection from world to image plane 

By knowing the pixel coordinates of Ihe image points p and the 3D coordinates of 
the corresponding scene points P, it is possible to compute the unknown 
parameters K,R t T and by solving Ihe perspective projection equation 





f=-^& 
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Camera Calibration (cont.) 



- We know that 



k[r\t] 



So there are 1 1 values to estimate: 
(the overall scale doesn't matter, so 
for example m 34 could be set to 1 } 



Each observed point gives us a pair of equations: 

A m 3 , -I- "^ i2 ■+- 7™%-*, + m 34 



To estimate 1 1 unknowns, we need at least 6 points to calibrate the 
camera *=£ solved using linear least squares 
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Camera Calibration (cont.) 



Having obtained the 3x4 projection matrix, it can be decomposed into the 
camera calibration matrix, the rotalion and posilion of the camera: k[k\t] 

Using standard techniques (QR decomposition) we can decompose the 
3x3 submatrix into the product of an upper triangular matrix K and a 

rotation matrix/? (orthogonal matrix) 

The translation T can subsequently be obtained by: 



T = K~ l 
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How do we measure distances with cameras? 



Impossible to capture 3D structure Irom a single image. We can only 
deduct the ray on which each image point lies. 



3D Object 




Lett Image 



Right Image 



■ Observe scene from 2 different viewpoints ^solve for the intersection of 
the rays and recover the 3D structure 
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Disparity in the human retina 
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How do we measure distances with cameras? 



Structure from stereo (Stereo-vision): 

^use two cameras with known relative position and orientation 



*%^ 



Structure from motion: 

i- use a single moving camera: both 3D structure and camera motion 
can be estimated up to a scale 



■ 
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Stereo Vision - The simplified case 



- The simplified case is an ideal case. It assumes that both cameras are 
identical and are aligned on a horizontal axis 



^ P = jX f ,Y y .Z p ) 




From Similar Triangles: 

Z P X p 

f _ »> 



z,= 



bf 



Z p b-X s 



Disparity 

difference in image location of the projection 
of a 3D point in two image planes 



Baseline 

distance between the optical centers of 
the two cameras 
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Stereo Vision facts 



z D = 



£>/ 



1 . Depth is inversely proportional to disparity (u { — u t ) 

■ Foreground objects have bigger disparity than background objects 



Disparity is proportional to stereo-baseline b 

■ The smaller Ihe baseline b the mare uncertain our estimate of depth 

■ However, as b is increased, some objects may appear in one camera, but not 
in the other (remember both cameras have parallel optical axes) 

The projections of a single 3D point onto Ihe left and the right stereo 
images are called 'correspondence pair' 



■ 
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Stereo Vision - the general case 



- Two identical cameras do not exist in nature! 

- Aligning both cameras on a horizontal axis is very hard — even in the most 
expensive stereo cameras! 




(R.T) 



In order to be able to use a stereo camera, we need the 

■ relative pose between the cameras (rotation, translation), and 

■ the focal length, image center, radial distortion of each 
o Use a calibration method, as mentioned before 
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Stereo Vision - the general case (cont.) 



■ To estimate the 3D position of P w we just construct the system ot equations 
of the left and right camera 




(ft,T) 



Left camera: 



set the world frame to coincide Pi 
with the left camera frame 



Right camera: j 7 — £ 
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Correspondence Problem 



Which patch in the Left image, corresponds to the projection of the same 3D 
scene point on the Right image? 

Correspondence search could be done by testing the query-patches at all pixel 
positions in the other image. Typical similarity measures are the Cross- 
Correlation and Sum of Squared Differences 

Exhaustive image search can be computationally very expensive! Is there a way 
to make the correspondence search 1 -dimensional? 
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Epipolar Geometry 



- Triangulaling 2 rays from image correspondences 

- Impose the epipolar constraint to aid matching: search for a 
correspondence along the epipolar line 

P = U, y, z) 

epipolar plaice 71 , 



Epipolar Line >-., 





Epipolar Line 2 



r pa m 



epipoles 

- The epipolar plane is defined by a 3D point P and the optical centers. 
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Correspondence Problem: Epipolar Constraint 



Thanks to the epipolar constraint, conjugate points can be searched 
along epipolar lines: this reduces the computational cost to 1 dimension 
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Epipolar Rectification 



■ Determines a transformation of each image plane so that pairs of 
conjugate epipolar lines become collinear and parallel 1o one ol the image 
axes (usually the horizontal one) 



Image from Left Camera Image from Right Camera 
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Epipolar Rectification (cont.) 



Determines a transformation of each image plane so that pairs of 
conjugate epipolar lines become collinear and parallel to one of the image 
axes (usually the horizontal one) 



Rotation 



mage from Left Camera Image from Right Camera 
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Epipolar Rectification (cont.) 



1 Determines a transformation of each image plane so that pairs of 
conjugate epipolar lines become collinear and parallel to one of the image 
axes (usually the horizontal one) 



Rotation 

1 

Focal lengths 



Image from Left Camera Image from Right Camera 
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Epipolar Rectification (cont.) 



- Determines a transformation of each image plane so that pairs of 
conjugate epipolar lines become collinear and parallel to one ol the image 
axes (usually the horizontal one) 



Rotation 

Focal lengths 
Lens Distortion 



Image from Left Camera 



Image from Right Camera 
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Epipolar Rectification (cont.) 



■ Determines a transformation of each image plane so that pairs of 
conjugate epipolar lines become collinear and parallel to one of the image 
axes (usually the horizontal one) 



Rotation 

H 
Focal lengths 

41 

Lens Distortion 

H 

Translation 



Image From Left Camerc 




Image from Right Camera 
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Epipolar Rectification (cont.) 



- Determines a transformation of each image plane so that pairs of 
conjugate epipolar lines become collinear and parallel to one of the image 
axes (usually the horizontal one) 



Rotatic 

Focal lengths 

Lens Distortion 

a 

Translation 



Image From Left Camera 



Image from Right Camera 

■ 
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Stereo Vision Output 2 - 3D Reconstruction via 
triangulation 



rr i^ ng ul^ te co rre-s po n d e n c e s t<z> 
cget 3D scr^n^ structure 



■ 



CIS Faculty 
Department of Information Systems 



Mansoura University 



Stereo Vision Output 2 - 3D Reconstruction via 
triangulation (cont.) 




Other Examples of 3D scene 
reconstruction from stereo 
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Stereo Vision Output 1 - Disparity map 



1 Find the correspondent points of all 
image pixels of the original images 

1 Compute the disparity for each pair 
of correspondences 

1 Disparity map: holds the disparity 
value at every pixel 

1 Usually visualized as grey-scale 
images. If lighter color corresponds 
to larger disparities, then objects 
closer to the camera appear lighter, 
than those further away. 




Disparity map 
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Stereo Vision - summary 



3D Object 




Left I mag*: 



Right Image 



1 . Stereo camera calibration <> compute camera relative pose 

2. Epipolar rectification ■=> align images & epipolar lines 

3. Search for correspondences 

4. Output: compute stereo triangulation or disparity map 

5. Consider baseline and image resolution to compute accuracy 



